Skip to content

refactor(cache): Integrate the Radix LRU cache with the --experimental-enable-optimized-metadata-cache flag#4872

Open
vedantdas-source wants to merge 4 commits into
radix-trie-cachefrom
radix-integrate
Open

refactor(cache): Integrate the Radix LRU cache with the --experimental-enable-optimized-metadata-cache flag#4872
vedantdas-source wants to merge 4 commits into
radix-trie-cachefrom
radix-integrate

Conversation

@vedantdas-source

@vedantdas-source vedantdas-source commented Jul 12, 2026

Copy link
Copy Markdown
Collaborator

Description

This PR integrates the Radix Tree based LRU cache with GCSFuse using the --experimental-enable-optimized-metadata-cache flag.

Changes Made

  • Added a new config field in bucket config and added logic to enable it in bucket_manager.go
  • Made appropriate changes to test files for benchmarking Radix Cache along with map cache

Benchmarking Results

goos: linux
goarch: amd64
pkg: github.com/googlecloudplatform/gcsfuse/v3/internal/cache/lru
cpu: Intel(R) Xeon(R) CPU @ 2.60GHz
BenchmarkLRU_Flat/Flat_MapLRU_Insert-48                          2864472       402.3 ns/op            87 B/op          1 allocs/op
BenchmarkLRU_Flat/Flat_RadixLRU_Insert-48                        1000000        1086 ns/op            81 B/op          2 allocs/op
BenchmarkLRU_Flat/Flat_MapLRU_Lookup-48                          5854327       201.7 ns/op             0 B/op          0 allocs/op
BenchmarkLRU_Flat/Flat_RadixLRU_Lookup-48                        1000000        1209 ns/op             0 B/op          0 allocs/op
BenchmarkLRU_Flat/Flat_MapLRU_ErasePrefix-48                          22    52187680 ns/op         35136 B/op          9 allocs/op
BenchmarkLRU_Flat/Flat_RadixLRU_ErasePrefix-48                     80914       15106 ns/op             0 B/op          0 allocs/op
BenchmarkLRU_Nested/Nested_MapLRU_Insert-48                      2678210       430.7 ns/op            91 B/op          1 allocs/op
BenchmarkLRU_Nested/Nested_RadixLRU_Insert-48                    1000000        1356 ns/op            81 B/op          2 allocs/op
BenchmarkLRU_Nested/Nested_MapLRU_Lookup-48                      5526685       212.6 ns/op             0 B/op          0 allocs/op
BenchmarkLRU_Nested/Nested_RadixLRU_Lookup-48                     818209        1458 ns/op             0 B/op          0 allocs/op
BenchmarkLRU_Nested/Nested_MapLRU_ErasePrefix-48                      21    53599548 ns/op         35136 B/op          9 allocs/op
BenchmarkLRU_Nested/Nested_RadixLRU_ErasePrefix-48                 79236       15344 ns/op             0 B/op          0 allocs/op
BenchmarkLRU_DeeplyNested/DeeplyNested_MapLRU_Insert-48          2475738       461.8 ns/op            96 B/op          1 allocs/op
BenchmarkLRU_DeeplyNested/DeeplyNested_RadixLRU_Insert-48        1000000        1372 ns/op            81 B/op          2 allocs/op
BenchmarkLRU_DeeplyNested/DeeplyNested_MapLRU_Lookup-48          4792593       250.1 ns/op             0 B/op          0 allocs/op
BenchmarkLRU_DeeplyNested/DeeplyNested_RadixLRU_Lookup-48        1000000        1367 ns/op             0 B/op          0 allocs/op
BenchmarkLRU_DeeplyNested/DeeplyNested_MapLRU_ErasePrefix-48          20    56798303 ns/op         35136 B/op          9 allocs/op
BenchmarkLRU_DeeplyNested/DeeplyNested_RadixLRU_ErasePrefix-48     79173       15964 ns/op            14 B/op          0 allocs/op
PASS
ok      github.com/googlecloudplatform/gcsfuse/v3/internal/cache/lru    101.715s

=== RUN   TestMapCacheWorkloads
    lru_memory_test.go:83: === WORKLOAD: flat (1000000 files) ===
    lru_memory_test.go:97: MapLRU               Heap Used:     126.69 MB
    lru_memory_test.go:111: RadixLRU             Heap Used:      91.75 MB
    lru_memory_test.go:83: === WORKLOAD: nested (1000000 files) ===
    lru_memory_test.go:97: MapLRU               Heap Used:     131.52 MB
    lru_memory_test.go:111: RadixLRU             Heap Used:      90.55 MB
    lru_memory_test.go:83: === WORKLOAD: deeply_nested (1000000 files) ===
    lru_memory_test.go:97: MapLRU               Heap Used:     134.55 MB
    lru_memory_test.go:111: RadixLRU             Heap Used:      85.91 MB
--- PASS: TestMapCacheWorkloads (7.57s)
PASS
ok      github.com/googlecloudplatform/gcsfuse/v3/internal/cache/lru    7.619s

Link to the issue in case of a bug fix.

Testing details

  1. Manual - NA
  2. Unit tests - NA
  3. Integration tests - NA

Any backward incompatible change? If so, please explain.

@vedantdas-source
vedantdas-source requested a review from a team as a code owner July 12, 2026 08:49
gemini-code-assist[bot]
gemini-code-assist Bot reviewed Jul 12, 2026
View reviewed changes

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for an optimized metadata cache using a radix-tree-based LRU cache (RadixLRU). It adds the EnableOptimizedMetadataCache configuration option to BucketConfig and integrates it into BucketManager to conditionally instantiate RadixLRU. It also includes comprehensive memory tests and benchmarks comparing RadixLRU with the standard MapLRU. The feedback suggests stopping the benchmark timer before resetting it in benchmarkErasePrefix to ensure consistent measurements across all iterations.

Comment thread internal/cache/lru/lru_workload_benchmark_test.go Outdated
@codecov

codecov Bot commented Jul 12, 2026 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 33.33333% with 4 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (radix-trie-cache@d83f47d). Learn more about missing BASE report.

Files with missing lines Patch % Lines
internal/gcsx/bucket_manager.go 40.00% 1 Missing and 2 partials ⚠️
cmd/mount.go 0.00% 1 Missing ⚠️
Additional details and impacted files 
@@                 Coverage Diff                 @@
##             radix-trie-cache    #4872   +/-   ##
===================================================
  Coverage                    ?   83.94%           
===================================================
  Files                       ?      174           
  Lines                       ?    21433           
  Branches                    ?        0           
===================================================
  Hits                        ?    17992           
  Misses                      ?     2764           
  Partials                    ?      677
Flag Coverage Δ
unittests 83.94% <33.33%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

vipnydav
vipnydav reviewed Jul 16, 2026
View reviewed changes
Comment thread internal/cache/metadata/statcache_memory_test.go Outdated
vipnydav
vipnydav reviewed Jul 16, 2026
View reviewed changes
Comment thread internal/cache/lru/lru_workload_benchmark_test.go Outdated
vipnydav
vipnydav reviewed Jul 16, 2026
View reviewed changes
Comment thread internal/cache/lru/lru_workload_benchmark_test.go
vipnydav
vipnydav reviewed Jul 16, 2026
View reviewed changes

@vipnydav vipnydav left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Left few minor comments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kislaykishore kislaykishore Awaiting requested review from kislaykishore
@vipnydav vipnydav Awaiting requested review from vipnydav
1 more reviewer
@gemini-code-assist gemini-code-assist[bot] gemini-code-assist[bot] left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@vedantdas-source @vipnydav